Electric power communication service identification method and system based on adaptive multi-channel attention

Abstract

The invention discloses a power communication service identification method and system based on adaptive multi-channel attention, and the method comprises the steps: taking a constructed power communication service identification network based on adaptive multi-channel attention as a teacher network, and effectively capturing diversified feature information in input data through a plurality of parallel feature channels; based on electric power communication service sample data, a teacher network is used to carry out adaptive knowledge distillation on an initial student network to obtain a final student network, a model can be compressed through knowledge distillation, the service identification speed is improved, the adopted adaptive knowledge distillation can dynamically adjust the knowledge transmission process along with the number of iterations, and the service identification efficiency is improved. According to the method, the lightweight student network can still maintain high recognition precision while remarkably reducing computing resource consumption, power communication network service recognition with high precision, self-adaptive adjustment and rapid reasoning is realized, and the power communication service recognition speed and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of power communication networks, and in particular to a method and system for identifying power communication services based on adaptive multi-channel attention. Background Art

[0002] With the continuous expansion of the scale and increasing complexity of the power system, the power communication network plays a vital role in ensuring the safe, stable and efficient operation of the power grid. However, the existing business identification and classification technologies still face many challenges. On the one hand, there are many types of business flows in the power communication network, and the data dimensions are diverse and complex. The traditional port-based and DPI (deep packet inspection) methods are difficult to effectively process the characteristics of high-dimensional and nonlinear data, resulting in low recognition accuracy. On the other hand, in actual scenarios, the traffic in the power communication network shows the characteristics of dynamic changes. The traditional methods cannot adaptively adjust the processing of data characteristics, resulting in uneven resource allocation and increasingly serious network congestion problems. In addition, with the introduction of new technologies such as big data and the Internet of Things, the business traffic and data volume that the power communication network needs to process have increased dramatically. The existing methods have shown obvious deficiencies in computing resource consumption and processing efficiency, and it is difficult to meet the requirements of modern power systems for business identification efficiency and robustness. Traditional methods show great limitations in dealing with complex data characteristics, dynamic network environments and high computing resource requirements. In particular, when facing the diverse business flows and changing network states in the power communication network, the existing technologies cannot provide sufficiently refined and efficient support.

[0003] Therefore, developing innovative power communication service identification methods that can be adaptively adjusted, efficiently utilize resources, and have high recognition accuracy has become the key to improving the overall performance of power communication networks. Power communication service identification requires extremely high real-time and accuracy. The existing models have complex network structures, resulting in slow training and inference speeds, and cannot meet the requirements of fast and accurate service identification. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for identifying power communication services based on adaptive multi-channel attention, which can improve the speed and accuracy of power communication service identification.

[0005] In order to solve the above technical problems, a technical solution adopted by the present invention is:

[0006] A method for identifying power communication services based on adaptive multi-channel attention comprises the following steps:

[0007] Constructing an adaptive multi-channel attention-based power communication service identification network, and using the adaptive multi-channel attention-based power communication service identification network as a teacher network;

[0008] Determining an initial student network based on the teacher network, and obtaining power communication service sample data;

[0009] Based on the electric power communication service sample data, the teacher network is used to perform adaptive knowledge distillation on the initial student network to obtain a final student network;

[0010] The power communication service flow to be identified is obtained, and the power communication service flow is input into the final student network for power communication service identification to obtain an identification result.

[0011] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0012] A power communication service identification system based on adaptive multi-channel attention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0013] Constructing an adaptive multi-channel attention-based power communication service identification network, and using the adaptive multi-channel attention-based power communication service identification network as a teacher network;

[0014] Determining an initial student network based on the teacher network, and obtaining power communication service sample data;

[0015] Based on the electric power communication service sample data, the teacher network is used to perform adaptive knowledge distillation on the initial student network to obtain a final student network;

[0016] The power communication service flow to be identified is obtained, and the power communication service flow is input into the final student network for power communication service identification to obtain an identification result.

[0017] The beneficial effects of the present invention are as follows: the constructed power communication business identification network based on adaptive multi-channel attention is used as a teacher network, and multiple parallel feature channels can effectively capture the diversified feature information in the input data, and then the initial student network is determined based on the teacher network, and the power communication business sample data is obtained, and the teacher network is used to perform adaptive knowledge distillation on the initial student network based on the power communication business sample data to obtain the final student network, and the model can be compressed through knowledge distillation to improve the speed of business identification, and the adaptive knowledge distillation used can dynamically adjust the knowledge transfer process with the number of iterations, so that the lightweight student network can significantly reduce the consumption of computing resources while still maintaining a high recognition accuracy, and the acquired power communication business flow is input into the final student network for power communication business identification to obtain the recognition result, realizing high-precision, adaptive adjustment and fast reasoning power communication network business identification, and improving the speed and accuracy of power communication business identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a method for identifying power communication services based on adaptive multi-channel attention according to an embodiment of the present invention;

[0019] Figure 2 It is a structural schematic diagram of a power communication service identification system based on adaptive multi-channel attention according to an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a power communication service identification network based on adaptive multi-channel attention in a power communication service identification method based on adaptive multi-channel attention according to an embodiment of the present invention;

[0021] Figure 4 Schematic diagram of adaptive knowledge distillation in a power communication service identification method based on adaptive multi-channel attention according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0023] Please refer to Figure 1 , a method for identifying power communication services based on adaptive multi-channel attention, comprising the steps of:

[0024] Constructing an adaptive multi-channel attention-based power communication service identification network, and using the adaptive multi-channel attention-based power communication service identification network as a teacher network;

[0025] Determining an initial student network based on the teacher network, and obtaining power communication service sample data;

[0026] Based on the electric power communication service sample data, the teacher network is used to perform adaptive knowledge distillation on the initial student network to obtain a final student network;

[0027] The power communication service flow to be identified is obtained, and the power communication service flow is input into the final student network for power communication service identification to obtain an identification result.

[0028] From the above description, it can be seen that the beneficial effects of the present invention are: the constructed power communication business identification network based on adaptive multi-channel attention is used as a teacher network, and multiple parallel feature channels can effectively capture the diversified feature information in the input data, and then the initial student network is determined based on the teacher network, and the power communication business sample data is obtained. The teacher network is used to perform adaptive knowledge distillation on the initial student network based on the power communication business sample data to obtain the final student network. The model can be compressed through knowledge distillation to improve the speed of business identification, and the adaptive knowledge distillation used can dynamically adjust the knowledge transfer process with the number of iterations, so that the lightweight student network can significantly reduce the consumption of computing resources while still maintaining a high recognition accuracy. The acquired power communication business flow is input into the final student network for power communication business identification to obtain the recognition result, realizing high-precision, adaptive adjustment and fast reasoning power communication network business identification, and improving the speed and accuracy of power communication business identification.

[0029] Furthermore, the construction of the power communication service identification network based on adaptive multi-channel attention includes:

[0030] Construct a single-channel power communication service identification network based on attention mechanism and interactive attention mechanism;

[0031] Based on the single-channel power communication service identification network, an adaptive multi-channel attention-based power communication service identification network is constructed.

[0032] From the above description, it can be seen that a single-channel power communication business identification network based on the attention mechanism and the interactive attention mechanism is first constructed, and then a power communication business identification network based on adaptive multi-channel attention is constructed based on the single-channel power communication business identification network. The power communication business identification network integrates the adaptive multi-channel mechanism, the attention mechanism and the interactive attention mechanism, which can effectively capture more comprehensive and diverse feature information of the input data, which is conducive to improving the recognition accuracy.

[0033] Furthermore, the construction of a single-channel power communication service identification network based on the attention mechanism and the interactive attention mechanism includes:

[0034] Generate an embedding layer, an attention layer, a first residual connection and a normalization layer, a mutual attention layer, a second residual connection and a normalization layer, a first linear layer, and a second linear layer connected in sequence;

[0035] A single-channel power communication service identification network based on attention mechanism and interactive attention mechanism is established according to the embedding layer, the attention layer, the first residual connection and normalization layer, the interactive attention layer, the second residual connection and normalization layer, the first linear layer and the second linear layer.

[0036] From the above description, it can be seen that the structure of the single-channel power communication service identification network based on the attention mechanism and the interactive attention mechanism includes an embedding layer, an attention layer, a first residual connection and a normalization layer, an interactive attention layer, a second residual connection and a normalization layer, a first linear layer, and a second linear layer connected in sequence, so that the model can dynamically focus on the most important part when processing the input data, thereby improving the model performance.

[0037] Further, the generating of the sequentially connected embedding layer, the attention layer, the first residual connection and normalization layer, the interactive attention layer, the second residual connection and normalization layer, the first linear layer and the second linear layer includes:

[0038] Generate an embedding layer, and set the dimension of the embedding layer to 512;

[0039] Generate an attention layer, a first residual connection and normalization layer, an interactive attention layer, and a second residual connection and normalization layer;

[0040] Generate a first linear layer and a second linear layer, and set the number of hidden neurons in the first linear layer to 2048 and the number of hidden neurons in the second linear layer to 512;

[0041] The embedding layer, the attention layer, the first residual connection and normalization layer, the interactive attention layer, the second residual connection and normalization layer, the first linear layer, and the second linear layer are connected in sequence.

[0042] From the above description, we can see that the higher the dimension of the embedding layer, the larger the feature space that the network can represent, and the better it can capture the complex features of the input data. The dimension of the embedding layer is set to 512, so that the embedding layer can provide more information to help the network better understand the input data. The hidden neurons of the first linear layer are set to 2048 and the hidden neurons of the second linear layer are set to 512, which can ensure the accuracy of network recognition.

[0043] Furthermore, the construction of an adaptive multi-channel attention-based power communication service identification network based on the single-channel power communication service identification network includes:

[0044] Generate an adaptive weight layer, a feature fusion layer, a third linear layer, and a Softmax layer connected in sequence;

[0045] The multiple single-channel power communication service identification networks are sorted in parallel, and the outputs of the multiple single-channel power communication service identification networks are fused, and then connected with the adaptive weight layer, the feature fusion layer, the third linear layer and the Softmax layer to obtain a power communication service identification network based on adaptive multi-channel attention.

[0046] From the above description, it can be seen that multiple single-channel power communication business identification networks are sorted in parallel, and the outputs of multiple single-channel power communication business identification networks are fused, and then connected with the adaptive weight layer, feature fusion layer, third linear layer and Softmax layer to obtain a power communication business identification network based on adaptive multi-channel attention. Different attention weights are assigned to the input data, so that the model can focus more on key information, significantly improve the efficiency and accuracy in processing complex data, and can enhance the feature representation capability.

[0047] Furthermore, the fusing of outputs of the plurality of single-channel power communication service identification networks comprises:

[0048]

[0049] In the formula, F fused represents the fused feature matrix, M represents the number of channels, α m Represents the adaptive weight corresponding to the mth channel, R m represents the feature representation of the mth channel.

[0050] From the above description, it can be seen that the outputs of multiple single-channel power communication service identification networks are fused in order to perform weighted summation of the features output by each channel according to adaptive weights to generate a unified feature representation for subsequent data processing.

[0051] Furthermore, the adaptive knowledge distillation of the initial student network using the teacher network based on the power communication service sample data to obtain the final student network includes:

[0052] Inputting the electric power communication service sample data into the teacher network, using the cross entropy loss function to calculate the loss between the predicted label and the true label of the teacher network, and back-propagating, updating the parameters of the teacher network, training multiple times until the teacher network converges and the value of the cross entropy loss function is minimized, thereby obtaining a trained teacher network;

[0053] In the first stage, the electric power communication service sample data is input into the initial student network, and the loss between the predicted label and the true label of the initial student network is calculated using a cross entropy loss function;

[0054] In the second stage, the knowledge of the trained teacher network is transferred to the initial student network using an adaptive knowledge distillation algorithm, the distillation loss is calculated according to the output of the trained teacher network and the output of the initial student network, and the total loss of the student network is calculated according to the distillation loss, the loss of the initial student network and the dynamic weight parameters. The training is repeated multiple times until the initial student network converges and the total loss is minimized to obtain the final student network.

[0055] From the above description, it can be seen that by implementing adaptive knowledge distillation, the knowledge of the teacher network is gradually transferred through two stages, and the transfer process is dynamically adjusted with the number of iterations to optimize the accuracy and efficiency of the student network, so that the final lightweight student network can significantly reduce the consumption of computing resources while still maintaining a high recognition accuracy.

[0056] Furthermore, the use of the cross entropy loss function to calculate the loss between the predicted label and the true label of the teacher network includes:

[0057]

[0058] In the formula, L represents the loss between the predicted label and the true label of the teacher network, N represents the number of samples, C represents the number of sample categories, and y i,j represents the true label of the i-th power communication service sample in the j-th category, It represents the probability that the teacher network predicts that the i-th power communication service sample is of the j-th category.

[0059] From the above description, we can see that the cross entropy loss function can directly measure the difference between the teacher network prediction and the true label. By minimizing the cross entropy loss, the prediction accuracy of the teacher network can be effectively improved. The cross entropy loss function conforms to the maximum likelihood estimation principle and can effectively solve the gradient vanishing problem, making the teacher network more stable during training.

[0060] Further, the use of the cross entropy loss function to calculate the loss between the predicted label and the true label of the initial student network includes:

[0061]

[0062] Where, L cls represents the loss between the predicted labels of the initial student network and the true labels, represents the true label of the i-th power communication service sample in the j-th category, represents the probability that the student network predicts that the i-th power communication service sample is of the j-th category, N represents the number of samples, and C represents the number of sample categories;

[0063] Calculating the distillation loss according to the output of the trained teacher network and the output of the initial student network includes:

[0064]

[0065] Where, L KD represents the distillation loss, T represents the distillation temperature, represents the probability that the teacher network predicts that the i-th power communication business sample is of the j-th category;

[0066] The calculating the total loss of the student network according to the distillation loss, the loss of the initial student network and the dynamic weight parameter comprises:

[0067] L total =λ·L cls +(1-λ)·L KD ;

[0068] Where, L total represents the total loss of the student network, and λ represents the dynamic weight parameter.

[0069] From the above description, it can be seen that calculating the total loss of the student network based on the distillation loss, the loss of the initial student network and the dynamic weight parameters can overcome the limitations of relying solely on the distillation loss for model training, better enable the student network to learn the knowledge of the teacher network, and ensure that the final student network can achieve fast and accurate power communication business identification.

[0070] Please refer to Figure 2 Another embodiment of the present invention provides an electric power communication service identification system based on adaptive multi-channel attention, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step in the above-mentioned electric power communication service identification method based on adaptive multi-channel attention is implemented.

[0071] The above-mentioned method and system for identifying power communication services based on adaptive multi-channel attention of the present invention can be applied to power communication service identification scenarios, and are described below through specific implementation methods:

[0072] Please refer to Figure 1 , Figure 3 and Figure 4 , Embodiment 1 of the present invention is:

[0073] A method for identifying power communication services based on adaptive multi-channel attention comprises the following steps:

[0074] S1. Constructing an adaptive multi-channel attention-based power communication service identification network, and using the adaptive multi-channel attention-based power communication service identification network as a teacher network, specifically including S11 to S13:

[0075] S11. Constructing a single-channel power communication service identification network based on the attention mechanism and the interactive attention mechanism, specifically including S111 to S112:

[0076] S111, generate an embedding layer, an attention layer, a first residual connection and a normalization layer, an interactive attention layer, a second residual connection and a normalization layer, a first linear layer, and a second linear layer that are connected in sequence.

[0077] Specifically, an embedding layer is generated, and the dimension of the embedding layer is set to 512; an attention layer, a first residual connection and normalization layer, an interactive attention layer, and a second residual connection and normalization layer are generated; a first linear layer and a second linear layer are generated, and the hidden neurons of the first linear layer are set to 2048, and the hidden neurons of the second linear layer are set to 512; the embedding layer, the attention layer, the first residual connection and normalization layer, the interactive attention layer, the second residual connection and normalization layer, the first linear layer, and the second linear layer are connected in sequence.

[0078] The embedding layer uses one-dimensional convolution to encode the features of the input samples. The size of the convolution kernel of the one-dimensional convolution is k, the step size is s, and the number of channels is C. The corresponding convolution layer is expressed as:

[0079] X′=Conv1D(X,W,b);

[0080] In the formula, X represents the input sample, represents the output features after convolution, represents the weight of the convolution kernel, Represents the bias. The activation function ReLU is applied after the convolution operation, and the maximum pooling is added, specifically:

[0081] E=MaxPooling1D(RELU(X′));

[0082] In the formula, represents the feature dimension after pooling, and d′ represents the feature length after the pooling operation.

[0083] The attention layer obtains the query component, key component and value component after linear mapping based on the input embedding feature E, specifically:

[0084] Q=EW q +b q ;

[0085] K=EW k +b k ;

[0086] V=EW v +b v ;

[0087] In the formula, represents the query component, represents the key component, Represents the value component, q m Represents the column vector of the query component matrix, k m Column vector representing the key component matrix, v m A column vector representing the value component matrix, represents the first learnable weight vector, represents the second learnable weight vector, represents the learnable third weight vector, b q represents the first bias term, b k represents the second bias term, b v Denotes the third bias term. Perform scaled dot product attention on Q, K, and V to calculate these local information, specifically:

[0088]

[0089] Where O represents the output of the attention layer.

[0090] The first residual connection and normalization layer performs a residual connection between the output O of the attention layer and the original input E, and normalizes the layer, specifically:

[0091] O′=LayerNorm(O+E);

[0092] Where O′ represents the output of the first residual connection and normalization layer.

[0093] The interactive attention layer captures high-order feature interactions through the output O′ of the first residual connection and the normalization layer through the AIN layer (Attention Interaction Network), specifically:

[0094]

[0095] In the formula, represents the feature interaction representation. The attention weight α of the interactive feature ij It is expressed as:

[0096] α ij =softmax(f(z ij )).

[0097] The output H of the AIN layer is expressed as:

[0098]

[0099] The second residual connection and normalization layer is similar to the first residual connection and normalization layer, and the output of the AIN layer is residually connected to the original input and normalized through the layer.

[0100] The first linear layer and the second linear layer are processed by a RELU activation function, specifically:

[0101] L = RELU(W lin Output norm +b lin );

[0102] In the formula, L represents the final output of the network, W lin Represents the weight matrix, Output norm represents the result of the output H of the AIN layer after the second residual connection and normalization layer, b lin Represents the bias vector.

[0103] S112, establishing a single-channel power communication service recognition network based on the attention mechanism and the interactive attention mechanism according to the embedding layer, the attention layer, the first residual connection and normalization layer, the interactive attention layer, the second residual connection and normalization layer, the first linear layer and the second linear layer, such as Figure 3 shown.

[0104] S12, constructing an adaptive multi-channel attention-based power communication service identification network based on the single-channel power communication service identification network, such as Figure 3 As shown, it specifically includes S121~S122:

[0105] S121, generate an adaptive weight layer, a feature fusion layer, a third linear layer and a Softmax layer that are connected in sequence.

[0106] The number of hidden neurons in the third linear layer is set to 2048.

[0107] S122. Arrange the multiple single-channel power communication service identification networks in parallel, fuse the outputs of the multiple single-channel power communication service identification networks, and connect them with the adaptive weight layer, the feature fusion layer, the third linear layer and the Softmax layer to obtain a power communication service identification network based on adaptive multi-channel attention.

[0108] Assume that the output of each channel is Where N represents the number of samples, and d′ represents the output dimension of the fully connected layer. For the output R of each channel, the global features of all channels are concatenated into a feature matrix G, which is:

[0109] G=[R 1 ; R 2 ;...;R m ];

[0110] In the formula, The weight h of each channel is generated through an MLP (Multilayer Perceptron) network with two hidden layers, specifically:

[0111] h = MLP(G);

[0112] In the formula, Finally, the adaptive weights of M channels are obtained through softmax, which are:

[0113] α = softmax(h);

[0114] In the formula, α={α 1 ,α 2 ,...,α m} represents the adaptive weight of each channel.

[0115] The step of arranging the plurality of single-channel power communication service identification networks in parallel comprises:

[0116] The three single-channel power communication service identification networks are arranged in parallel.

[0117] The features of all channels are combined through weighted fusion operation and input into the fully connected layer and the Softmax layer to finally generate the classification result. The goal of weighted fusion is to weight the features output by each channel according to the adaptive weights to generate a unified feature representation. The output fusion of multiple single-channel power communication service identification networks includes:

[0118]

[0119] In the formula, represents the fused feature matrix, M represents the number of channels, α m Represents the adaptive weight corresponding to the mth channel, R m Represents the feature representation of the mth channel. The weighted fusion feature F fused Further feature extraction is performed through the fully connected layer, specifically:

[0120] F = RELU (F fused ·W f +bf );

[0121] In the formula, represents the weight matrix of the fully connected layer, represents the bias term, d out represents the dimension of the output feature, and F represents the extracted feature.

[0122] S13. Use the adaptive multi-channel attention-based power communication service identification network as a teacher network.

[0123] S2. Determine an initial student network based on the teacher network, and obtain power communication service sample data.

[0124] Among them, the structure of the initial student network is an embedding layer, an attention layer, a first residual connection and a normalization layer, an interactive attention layer, a second residual connection and a normalization layer, a first linear layer, and a second linear layer connected in sequence, the dimension of the embedding layer is set to 256, the dimension of the attention layer is set to 256, the dimension of the interactive attention layer is set to 256, the hidden neurons of the first linear layer are set to 1024, and the hidden neurons of the second linear layer are set to 256. It can be seen that the initial student network is a simplification of the teacher network.

[0125] S3, based on the power communication service sample data, use the teacher network to perform adaptive knowledge distillation on the initial student network to obtain a final student network, such as Figure 4 As shown, it specifically includes S31 to S33:

[0126] S31. Input the electric power communication service sample data into the teacher network, use the cross entropy loss function to calculate the loss between the predicted label and the true label of the teacher network, and back propagate, update the parameters of the teacher network, and train multiple times until the teacher network converges and the value of the cross entropy loss function is minimized, thereby obtaining the trained teacher network.

[0127] The method of calculating the loss between the predicted label and the true label of the teacher network by using the cross entropy loss function includes:

[0128]

[0129] In the formula, L represents the loss between the predicted label and the true label of the teacher network, N represents the number of samples, C represents the number of sample categories, and y i,j represents the true label of the i-th power communication service sample in the j-th category, It represents the probability that the teacher network predicts that the i-th power communication service sample is of the j-th category.

[0130] S32. In the first stage, the electric power communication service sample data is input into the initial student network, and the loss between the predicted label and the true label of the initial student network is calculated using a cross entropy loss function.

[0131] The method of calculating the loss between the predicted label and the true label of the initial student network by using the cross entropy loss function includes:

[0132]

[0133] Where, L cls represents the loss between the predicted labels of the initial student network and the true labels, represents the true label of the i-th power communication service sample in the j-th category, It represents the probability that the student network predicts that the i-th power communication service sample is of the j-th category.

[0134] S33. In the second stage, the knowledge of the trained teacher network is transferred to the initial student network using an adaptive knowledge distillation algorithm, the distillation loss is calculated according to the output of the trained teacher network and the output of the initial student network, and the total loss of the student network is calculated according to the distillation loss, the loss of the initial student network and the dynamic weight parameters. The training is repeated multiple times until the initial student network converges and the total loss is minimized to obtain the final student network.

[0135] The step of calculating the distillation loss according to the output of the trained teacher network and the output of the initial student network comprises:

[0136]

[0137] Where, L KD represents the distillation loss, T represents the distillation temperature, It represents the probability that the teacher network predicts that the i-th power communication business sample is of the j-th category.

[0138] Knowledge distillation is a technique for compressing deep learning models. It can extract the knowledge of the "teacher" model and then transfer this knowledge to a smaller and lighter student model. In traditional knowledge distillation methods, the predicted output of the teacher model is used as soft labels, and the student model learns the knowledge of the teacher model by minimizing the loss with these soft labels. Power communication service identification requires extremely high real-time and accuracy. The existing models have slow training and inference speeds due to the complex network structure and cannot meet the requirements of fast service identification. Knowledge distillation can be used to compress the model and improve the speed and accuracy of service identification. However, with the increase in task complexity and the expansion of deep learning network structures, simple knowledge transfer methods cannot fully utilize the rich information in the teacher model. In order to fully utilize the rich information in the teacher model, the present invention proposes an adaptive knowledge distillation framework for compressing the power communication service identification model into a lightweight model to improve the speed and accuracy of service identification.

[0139] The adaptive knowledge distillation algorithm adjusts the different attention paid to the two loss functions through the weight parameter λ. Specifically, the adaptive knowledge distillation algorithm operates on the loss of the model output layer and controls and switches the stage through the weight of the loss layer. λ represents the current degree of attention paid to the corresponding task. The weight of the loss term has a great influence on the learning process. Tasks with large loss values ​​are updated faster than tasks with small loss values. In the first stage, λ is set to 1, and only the learning task of the original data is performed. Because the weight of the tasks in other stages is 0, the back propagation of their losses is stopped. The parameter update of the student network in the first stage is expressed as:

[0140]

[0141] In the formula, Δw 1 represents the updated student network parameters in the first stage, w 1 Represents the student network parameters before the first stage update.

[0142] In the second stage, according to the number of training rounds, λ gradually decreases until it reaches 0. The purpose is to learn the original data and the predicted data at the same time, and gradually transfer the knowledge of the teacher network to the student network. λ is expressed as:

[0143]

[0144] In the formula, B represents the current training round, B 1 represents the number of training rounds in the first stage, B 2 represents the number of training rounds in the second stage, B max Indicates the maximum number of training rounds set. Once the tasks in the previous stage are trained, the attention to the subsequent tasks will increase rapidly.

[0145] The calculating the total loss of the student network according to the distillation loss, the loss of the initial student network and the dynamic weight parameter comprises:

[0146] L total =λ·L cls +(1-λ)·L KD ;

[0147] Where, L total represents the total loss of the student network, and λ represents the dynamic weight parameter.

[0148] The parameter update of the student network in the second stage is expressed as:

[0149]

[0150] In the formula, Δw 2 represents the updated student network parameters in the second stage, w 2 Represents the student network parameters before the second stage update.

[0151] S4. Obtain the power communication service flow to be identified, and input the power communication service flow into the final student network to perform power communication service identification to obtain an identification result.

[0152] Among them, the identification result is the type of power communication service. In an optional implementation, the identification result includes dispatch telephone, power metering, dispatch automation, relay protection, protection management system, safety and stability management system, video conferencing, video surveillance, administrative telephone, monitoring system, financial marketing information or administrative management, as shown in Table 1.

[0153] Table 1 Types of power communication services

[0154]

[0155] Aiming at the diversified business flows and constantly changing network states in the power communication network, the present invention integrates the adaptive multi-channel mechanism, the attention mechanism and the interactive attention mechanism, effectively captures the diversified feature information in the input data through multiple parallel feature channels, and introduces an adaptive weight mechanism to realize dynamic weighted adjustment of the contributions of different channels. The knowledge of the teacher network is gradually transferred through two stages through an adaptive knowledge distillation method, and the transfer process is dynamically adjusted with the number of iterations, so that the lightweight student network can maintain a high recognition accuracy while significantly reducing the consumption of computing resources. In this way, the power communication network business identification with high recognition accuracy, adaptive adjustment and fast reasoning is realized, which is particularly suitable for intelligent recognition application scenarios with complex data characteristics, dynamic network environment and high computing resource requirements.

[0156] Please refer to Figure 2, Embodiment 2 of the present invention is:

[0157] A power communication service identification system based on adaptive multi-channel attention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step in the above-mentioned power communication service identification method based on adaptive multi-channel attention is implemented.

[0158] In summary, the present invention provides a method and system for identifying electric power communication services based on adaptive multi-channel attention, wherein the constructed electric power communication service identification network based on adaptive multi-channel attention is used as a teacher network, and multiple parallel feature channels can effectively capture diversified feature information in the input data, and then the initial student network is determined based on the teacher network, and the electric power communication service sample data is obtained, and the teacher network is used to perform adaptive knowledge distillation on the initial student network based on the electric power communication service sample data to obtain the final student network, and the model can be compressed through knowledge distillation to improve the speed of service identification, and the adaptive knowledge distillation used can dynamically adjust the knowledge transfer process with the number of iterations, so that the lightweight student network can significantly reduce the consumption of computing resources while still maintaining a high recognition accuracy, and the acquired electric power communication service flow is input into the final student network for electric power communication service identification to obtain the identification. As a result, high-precision, adaptive adjustment and fast inference power communication network business identification is achieved, and the speed and accuracy of power communication business identification are improved; in addition, a single-channel power communication business identification network based on attention mechanism and interactive attention mechanism is first constructed, and then a power communication business identification network based on adaptive multi-channel attention is constructed based on the single-channel power communication business identification network, so that the power communication business identification network integrates adaptive multi-channel mechanism, attention mechanism and interactive attention mechanism, which can effectively capture more comprehensive and diverse feature information of input data, which is conducive to improving recognition accuracy; and, by implementing adaptive knowledge distillation, the knowledge of the teacher network is gradually transferred through two stages, and the transfer process is dynamically adjusted with the number of iterations, the accuracy and efficiency of the student network are optimized, so that the final lightweight student network can significantly reduce the consumption of computing resources while still maintaining a high recognition accuracy.

[0159] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for identifying power communication services based on adaptive multi-channel attention, characterized in that: Includes steps: Constructing an adaptive multi-channel attention-based power communication service identification network, and using the adaptive multi-channel attention-based power communication service identification network as a teacher network; Determining an initial student network based on the teacher network, and obtaining power communication service sample data; Based on the electric power communication service sample data, the teacher network is used to perform adaptive knowledge distillation on the initial student network to obtain a final student network; The power communication service flow to be identified is obtained, and the power communication service flow is input into the final student network for power communication service identification to obtain an identification result.

2. According to claim 1, a method for identifying power communication services based on adaptive multi-channel attention is characterized in that: The construction of the power communication service identification network based on adaptive multi-channel attention includes: Construct a single-channel power communication service identification network based on attention mechanism and interactive attention mechanism; Based on the single-channel power communication service identification network, an adaptive multi-channel attention-based power communication service identification network is constructed.

3. According to claim 2, a method for identifying power communication services based on adaptive multi-channel attention is characterized in that: The construction of a single-channel power communication service identification network based on the attention mechanism and the interactive attention mechanism includes: Generate an embedding layer, an attention layer, a first residual connection and a normalization layer, a mutual attention layer, a second residual connection and a normalization layer, a first linear layer, and a second linear layer connected in sequence; A single-channel power communication service identification network based on attention mechanism and interactive attention mechanism is established according to the embedding layer, the attention layer, the first residual connection and normalization layer, the interactive attention layer, the second residual connection and normalization layer, the first linear layer and the second linear layer.

4. According to claim 3, a method for identifying power communication services based on adaptive multi-channel attention is characterized in that: The generating of the sequentially connected embedding layer, the attention layer, the first residual connection and normalization layer, the interactive attention layer, the second residual connection and normalization layer, the first linear layer and the second linear layer comprises: Generate an embedding layer, and set the dimension of the embedding layer to 512; Generate an attention layer, a first residual connection and normalization layer, an interactive attention layer, and a second residual connection and normalization layer; Generate a first linear layer and a second linear layer, and set the number of hidden neurons in the first linear layer to 2048 and the number of hidden neurons in the second linear layer to 512; The embedding layer, the attention layer, the first residual connection and normalization layer, the interactive attention layer, the second residual connection and normalization layer, the first linear layer, and the second linear layer are connected in sequence.

5. According to claim 2, a method for identifying power communication services based on adaptive multi-channel attention is characterized in that: The method of constructing an adaptive multi-channel attention-based power communication service identification network based on the single-channel power communication service identification network includes: Generate an adaptive weight layer, a feature fusion layer, a third linear layer, and a Softmax layer connected in sequence; The multiple single-channel power communication service identification networks are sorted in parallel, and the outputs of the multiple single-channel power communication service identification networks are fused, and then connected with the adaptive weight layer, the feature fusion layer, the third linear layer and the Softmax layer to obtain a power communication service identification network based on adaptive multi-channel attention.

6. The method for identifying power communication services based on adaptive multi-channel attention according to claim 5, characterized in that: The step of fusing the outputs of the plurality of single-channel power communication service identification networks comprises: In the formula, F fused represents the fused feature matrix, M represents the number of channels, α m Represents the adaptive weight corresponding to the mth channel, R m represents the feature representation of the mth channel.

7. The method for identifying power communication services based on adaptive multi-channel attention according to claim 1, characterized in that: The adaptive knowledge distillation of the initial student network using the teacher network based on the power communication service sample data to obtain the final student network includes: Inputting the electric power communication service sample data into the teacher network, using the cross entropy loss function to calculate the loss between the predicted label and the true label of the teacher network, and back-propagating, updating the parameters of the teacher network, training multiple times until the teacher network converges and the value of the cross entropy loss function is minimized, thereby obtaining a trained teacher network; In the first stage, the electric power communication service sample data is input into the initial student network, and the loss between the predicted label and the true label of the initial student network is calculated using a cross entropy loss function; In the second stage, the knowledge of the trained teacher network is transferred to the initial student network using an adaptive knowledge distillation algorithm, the distillation loss is calculated according to the output of the trained teacher network and the output of the initial student network, and the total loss of the student network is calculated according to the distillation loss, the loss of the initial student network and the dynamic weight parameters. The training is repeated multiple times until the initial student network converges and the total loss is minimized to obtain the final student network.

8. The method for identifying power communication services based on adaptive multi-channel attention according to claim 7, characterized in that: The method of calculating the loss between the predicted label and the true label of the teacher network by using the cross entropy loss function includes: In the formula, L represents the loss between the predicted label and the true label of the teacher network, N represents the number of samples, C represents the number of sample categories, and y i,j represents the true label of the i-th power communication service sample in the j-th category, It represents the probability that the teacher network predicts that the i-th power communication service sample is of the j-th category.

9. The method for identifying power communication services based on adaptive multi-channel attention according to claim 7, characterized in that: The use of the cross entropy loss function to calculate the loss between the predicted label and the true label of the initial student network includes: Where, L cls represents the loss between the predicted labels of the initial student network and the true labels, represents the true label of the i-th power communication service sample in the j-th category, represents the probability that the student network predicts that the i-th power communication service sample is of the j-th category, N represents the number of samples, and C represents the number of sample categories; Calculating the distillation loss according to the output of the trained teacher network and the output of the initial student network includes: Where, L KD represents the distillation loss, T represents the distillation temperature, represents the probability that the teacher network predicts that the i-th power communication business sample is of the j-th category; The calculating the total loss of the student network according to the distillation loss, the loss of the initial student network and the dynamic weight parameter comprises: L total =λ·L cls +(1-λ)·L KD ; Where, L total represents the total loss of the student network, and λ represents the dynamic weight parameter.

10. A power communication service identification system based on adaptive multi-channel attention, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, each step of the method for identifying power communication services based on adaptive multi-channel attention according to any one of claims 1 to 9 is implemented.

Images