Lightweight Method and System for Overhead Line Power Outage Prediction Neural Network

Abstract

This invention discloses a lightweight method and system for predicting overhead line power outages using a neural network, belonging to the field of distribution network big data technology. The method includes: acquiring an overhead line power outage dataset; training a large neural network model on the dataset; using the dataset as input to the lightweight neural network model and fusing the output features of the first i layers of the large neural network model to supervise the output features of the i-th layer of the lightweight neural network model; fitting the output features of the last layer of the lightweight neural network model to the output features of the last layer of the large neural network model, performing knowledge distillation on the lightweight neural network model to obtain the target neural network model. This invention achieves a better balance between algorithm performance and model size, resulting in a lightweight model with improved performance.

Description

Technical Field

[0001] This invention relates to the field of big data technology for power distribution networks, specifically to a lightweight method and system for predicting power outages on overhead lines using neural networks. Background Technology

[0002] With economic and social development, the coverage of power grid substations and distribution networks continues to expand, and the impact of severe weather such as strong winds, lightning, and smog on the safe operation of the power grid is increasing. Under the influence of extreme weather, 10kV overhead distribution lines are more prone to tripping, causing regional power grid line failures and outages, affecting the normal production and life of residents.

[0003] Currently, there is a considerable amount of research both domestically and internationally on the risk assessment and prevention of meteorological disasters affecting power grids, mainly focusing on the impact of weather events such as snow and strong winds on distribution networks. Current research on overhead line outage prediction primarily relies on training machine learning or deep learning models using large-scale data to obtain outage prediction models, which are then deployed to a central processing cluster to achieve overhead line outage prediction.

[0004] Training requires complex models and significant computational resources to extract information from massive, highly redundant datasets. In experiments, the best-performing models are often large-scale, sometimes even ensembles of multiple models. However, large-scale models are inconvenient to deploy in services, with common bottlenecks including slow inference speed and high resource requirements (memory, GPU memory, etc.). Training excessively small deep learning models directly can lead to underfitting and difficulty in extracting information from large datasets. Therefore, balancing algorithm performance and model size is crucial for deploying overhead power line outage prediction models in production environments.

[0005] In related technologies, Chinese invention patent application CN113987953A discloses a method for lightweighting a fault detection model. This method trains a first power equipment fault detection model using a preset training set and a preset test set to obtain a second power equipment fault detection model. A lightweight network structure is then used to replace the basic network structure of the second power equipment fault detection model to obtain a third power equipment fault detection model. The third power equipment fault detection model is then input into a preset compression and fine-tuning algorithm for compression and fine-tuning to obtain a fourth power equipment fault detection model. Finally, the fourth power equipment fault detection model is input into a preset optimization engine to optimize it, resulting in a lightweight power equipment fault detection model. This method reduces the structural complexity of the neural network model while ensuring the accuracy of the output results.

[0006] However, it extracts the distance between the teacher model and the student model at the output layer using a pre-defined knowledge distillation algorithm as the loss function, and then transfers the features of the teacher model to the student model based on the loss function. Therefore, its approach directly uses the final logits, i.e., the output of the final fully connected layer, for supervision. The information output from the intermediate layers of the teacher model is not fully utilized by the student model, thus limiting the performance of the student model. Summary of the Invention

[0007] The technical problem to be solved by this invention is how to better balance algorithm performance and model size while ensuring that the student model has good performance.

[0008] The present invention solves the above-mentioned technical problems through the following technical means:

[0009] On one hand, this invention proposes a lightweight method for predicting overhead line power outages using a neural network, the method comprising:

[0010] Obtain the overhead power outage dataset;

[0011] A large neural network model was trained on the aforementioned overhead line power outage dataset.

[0012] The overhead line outage dataset is used as input to a lightweight neural network model, and the output features of the first i layers of the ultra-large neural network model are fused together to supervise the output features of the i-th layer of the lightweight neural network model.

[0013] The output features of the last layer of the lightweight neural network model are fitted to the output features of the last layer of the ultra-large neural network model, and knowledge distillation is performed on the lightweight neural network model to obtain the target neural network model.

[0014] This invention trains a massive overhead power line outage prediction neural network model using a large dataset of overhead power line outages. Then, based on certain network size constraints, a lightweight neural network model suitable for practical deployment is designed. Finally, using an early knowledge fusion distillation algorithm, the parameters of the massive neural network model are fixed on the training set of the target scenario. Inference is performed using both the massive model and the lightweight model, allowing the output of the lightweight neural network model to fit the processed output of the massive model. Knowledge distillation reduces the learning difficulty of the lightweight neural network model, enabling better training and transferring the predictive power of the massive model to the lightweight model, resulting in superior performance. Furthermore, the model distillation algorithm based on a multi-layer early knowledge fusion mechanism fully utilizes the output of the intermediate layers of the massive neural network model to supervise the learning of the lightweight neural network model, further improving its performance.

[0015] Furthermore, the overhead line power outage data dataset centrally includes meteorological feature data, geographical feature data, and power grid feature data;

[0016] The meteorological characteristic data includes the maximum intensity of the severe convective radar, the lightning current value, and the number of lightning return strokes.

[0017] The geographic feature data includes elevation, slope, and aspect;

[0018] The power grid characteristic data includes the number of overloads, the number of heavy loads, and the number of defects.

[0019] Furthermore, training a large neural network model on the overhead line outage dataset includes:

[0020] The overhead line power outage dataset is subjected to dimension unification processing to obtain dimension-unified data;

[0021] The ultra-large neural network model is trained using the dimensional uniform data, and residual connection and batch normalization operations are used during the training phase to obtain an ultra-large-scale neural network for predicting overhead line power outages.

[0022] The loss function used for model training is the binary cross-entropy loss function based on data label supervision, expressed by the following formula:

[0023]

[0024] In the formula: y i p represents the true fault label of sample i, with 1 for positive class and 0 for negative class. i Let N represent the probability that sample i is predicted to be of the positive class, and N represent the total number of samples.

[0025] Furthermore, the ultra-large neural network model includes a first input layer, a first backbone network, and a first prediction network, wherein the output features of the first input layer serve as the input of the first backbone network, and the output features of the first backbone network serve as the input of the first prediction network;

[0026] The first backbone network includes a multi-layer neural network, with adjacent layers connected by a residual network. The first prediction network includes a fully connected layer and a softmax classifier, with the output of the fully connected layer connected to the softmax classifier.

[0027] The first backbone network has a depth of 18 layers, with 256 neurons in each layer.

[0028] Furthermore, the loss function of the lightweight neural network model is:

[0029] L = L bce +λL MKD_F

[0030] Among them, L bce L is a binary cross-entropy loss function based on data label supervision. MKD_F The loss function for multi-layer early knowledge fusion distillation is λ, where λ is the weight.

[0031]

[0032]

[0033] Among them, y i p represents the true fault label of sample i, with 1 for positive class and 0 for negative class. i Let represent the probability that sample i is predicted as positive, N represent the total number of samples, i is the layer index of the lightweight neural network model, j is the index of the layers of all ultra-large neural network models before the i-th layer, and D is the knowledge distillation loss function. These are the output features of the intermediate layer of the lightweight neural network model. These are the output features of the intermediate layers of the aforementioned large neural network model.

[0034] Furthermore, the lightweight neural network model includes a second input layer, a second backbone network, and a second prediction network. The output features of the second input layer serve as the input to the second backbone network, and the output features of the second backbone network serve as the input to the second prediction network.

[0035] The second backbone network includes a multi-layer neural network, with adjacent layers connected by a residual network. The second prediction network includes a fully connected layer and a softmax classifier, with the output of the fully connected layer connected to the softmax classifier.

[0036] The second backbone network has a depth of 5 layers, with 64 neurons in each layer.

[0037] Furthermore, this invention also proposes a lightweight neural network system for predicting overhead line power outages, the system comprising:

[0038] The acquisition module is used to acquire overhead line power outage datasets;

[0039] The first training module is used to train a large neural network model on the overhead line outage dataset.

[0040] The supervision module is used to take the overhead line power outage dataset as input to the lightweight neural network model and fuse the output features of the first i layers of the ultra-large neural network model to supervise the output features of the i-th layer of the lightweight neural network model.

[0041] The knowledge distillation module is used to fit the output features of the last layer of the lightweight neural network model to the output features of the last layer of the ultra-large neural network model, and to perform knowledge distillation on the lightweight neural network model to obtain the target neural network model.

[0042] Furthermore, the overhead line power outage data dataset centrally includes meteorological feature data, geographical feature data, and power grid feature data;

[0043] The meteorological characteristic data includes the maximum intensity of the severe convective radar, the lightning current value, and the number of lightning return strokes.

[0044] The geographic feature data includes elevation, slope, and aspect;

[0045] The power grid characteristic data includes the number of overloads, the number of heavy loads, and the number of defects.

[0046] Furthermore, the ultra-large neural network model includes a first input layer, a first backbone network, and a first prediction network, wherein the output features of the first input layer serve as the input of the first backbone network, and the output features of the first backbone network serve as the input of the first prediction network;

[0047] The first backbone network includes a multi-layer neural network, with adjacent layers connected by a residual network. The first prediction network includes a fully connected layer and a softmax classifier, with the output of the fully connected layer connected to the softmax classifier.

[0048] The first backbone network has a depth of 18 layers, with 256 neurons in each layer.

[0049] Furthermore, the loss function of the lightweight neural network model is:

[0050] L = L bce +λL MKD_F

[0051] Among them, L bce L is a binary cross-entropy loss function based on data label supervision. MKD_F The loss function for multi-layer early knowledge fusion distillation is λ, where λ is the weight.

[0052]

[0053]

[0054] Among them, y i p represents the true fault label of sample i, with 1 for positive class and 0 for negative class. iLet represent the probability that sample i is predicted as positive, N represent the total number of samples, i is the layer index of the lightweight neural network model, j is the index of the layers of all ultra-large neural network models before the i-th layer, and D is the knowledge distillation loss function. These are the output features of the intermediate layer of the lightweight neural network model. These are the output features of the intermediate layers of the aforementioned large neural network model.

[0055] The advantages of this invention are:

[0056] (1) This invention utilizes a massive overhead power outage dataset to train an ultra-large-scale overhead power outage prediction neural network model. Then, based on certain network size constraints, a lightweight neural network model suitable for practical deployment is designed. Finally, using an early knowledge fusion distillation algorithm, the parameters of the ultra-large neural network model are fixed on the training set of the target scenario. Simultaneously, the ultra-large model and the lightweight model are used for inference, allowing the output of the lightweight neural network model to fit the output of the processed ultra-large model. Through knowledge distillation, the learning difficulty of the lightweight neural network model is reduced, enabling better training of the lightweight model and transferring the predictive power of the ultra-large model to the lightweight model, thus giving the lightweight network model superior performance. Moreover, the model distillation algorithm based on a multi-layer early knowledge fusion mechanism fully utilizes the output of the intermediate layers of the ultra-large neural network model to supervise the learning of the lightweight neural network model, further improving the performance of the lightweight neural network model.

[0057] (2) The training dataset constructed in this invention includes meteorological feature data, geographical feature data and power grid feature data. The meteorological features include the maximum intensity of strong convection radar, the magnitude of lightning current and the number of lightning return strokes, so as to predict the power outage of overhead lines under strong current weather.

[0058] (3) Residual networks are used to connect adjacent neural network layers in the neural network structure so that the later layers of the deep network are learned to be identity mapping models and degenerate into shallow networks. This greatly reduces the optimization difficulty for deep networks.

[0059] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating a lightweight method for predicting overhead line power outages using a neural network, according to an embodiment of the present invention.

[0061] Figure 2 This is a schematic diagram of the traditional knowledge distillation principle in one embodiment of the present invention;

[0062] Figure 3 This is a schematic diagram of the principle of distillation based on early knowledge fusion in one embodiment of the present invention;

[0063] Figure 4 This is a schematic diagram of a distillation model based on an early knowledge fusion mechanism in one embodiment of the present invention;

[0064] Figure 5 This is a schematic diagram of the structure of a super-large neural network model in one embodiment of the present invention;

[0065] Figure 6 This is a schematic diagram of the structure of a lightweight neural network model in one embodiment of the present invention;

[0066] Figure 7 This is a schematic diagram of the lightweight system of the overhead line power outage prediction neural network in another embodiment of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] like Figure 1 As shown, the first embodiment of the present invention proposes a lightweight method for predicting overhead line power outages using a neural network, the method comprising the following steps:

[0069] S10. Obtain the overhead line power outage dataset.

[0070] S20. Train a large neural network model on the overhead line outage dataset.

[0071] S30. The overhead line outage dataset is used as input to the lightweight neural network model, and the output features of the first i layers of the ultra-large neural network model are fused together to supervise the output features of the i-th layer of the lightweight neural network model.

[0072] S40. Fit the output features of the last layer of the lightweight neural network model to the output features of the last layer of the ultra-large neural network model, perform knowledge distillation on the lightweight neural network model, and obtain the target neural network model.

[0073] It should be noted that this embodiment utilizes a knowledge distillation algorithm to transfer the prediction capabilities of a large-scale overhead line outage prediction neural network to a lightweight overhead line outage prediction neural network. Knowledge distillation uses a Teacher-Student model, where the teacher model is the output of the "overhead line outage prediction capability," and the student model is the receiver of the "overhead line outage prediction capability."

[0074] The process of knowledge distillation consists of two stages:

[0075] (1) Training of ultra-large neural network models: Training of ultra-large neural network models is called Teacher model. The characteristics of ultra-large neural network models are that the models are relatively complex and can also be integrated from multiple separately trained models.

[0076] This embodiment does not impose any restrictions on the Teacher model regarding model architecture, number of parameters, or whether it is integrated. It only requires that for any input X, it can output Y. After the softmax layer is mapped, the output value is the probability value of the corresponding category.

[0077] (2) Lightweight Neural Network Model Knowledge Distillation: This refers to training a Student model. A lightweight neural network model is a single model with a small number of parameters and a relatively simple model structure. Similarly, for any input X, it can output Y, and Y, after being mapped by a softmax layer, can also output the probability value corresponding to the corresponding category.

[0078] like Figures 2 to 4 As shown, unlike traditional knowledge distillation, this embodiment utilizes knowledge distillation to transfer the predictive power of a large model to a lightweight model, achieving a better balance between algorithm performance and model size. This allows the overhead power outage prediction neural network model to be deployed to a central computing cluster for power outage prediction. Furthermore, to enable the lightweight neural network model to learn more useful knowledge, an early knowledge fusion mechanism is designed. This mechanism uses the outputs of the large neural network model in multiple early stages to supervise the outputs of the student model at each stage. By reducing the learning difficulty of the lightweight model through the early knowledge fusion distillation method, the lightweight model can be trained better, resulting in a higher-performance lightweight model.

[0079] In one embodiment, the overhead line power outage data acquired in step S10 includes meteorological feature data, geographical feature data, and power grid feature data;

[0080] The meteorological characteristic data includes the maximum intensity of the severe convective radar, the lightning current value, and the number of lightning return strokes.

[0081] The geographic feature data includes elevation, slope, and aspect;

[0082] The power grid characteristic data includes the number of overloads, the number of heavy loads, and the number of defects.

[0083] It should be noted that, in order to obtain the feature information required for overhead line power outage prediction, this embodiment collected big data information from tens of thousands of overhead lines in many regions over the past two years, including meteorological information, geographical information and power grid status information. From this data, feature information and corresponding labels required for training and verification were extracted to generate 1 million data samples for algorithm training and verification.

[0084] In this embodiment, the total data samples are divided into training and validation datasets in a 9:1 ratio, with the training set having 900,000 samples and the test set having 100,000 samples.

[0085] In one embodiment, step S20 specifically includes the following steps:

[0086] S21. Perform dimensional unification processing on the overhead line power outage dataset to obtain dimensional unified data.

[0087] Furthermore, the meteorological feature data in the overhead line power outage dataset includes the maximum intensity of strong convection radar f1, the magnitude of lightning current f2, and the number of lightning return strokes f3; the geographical feature data includes elevation l1, slope l2, and aspect l3; and the power grid feature data includes overload count n1, heavy load count n2, and defect count n3. Because these data have inconsistent dimensions, which is unfavorable for neural network training, this embodiment uses the following process to unify the dimensions of the data:

[0088] x′=(xx mean ) / x std

[0089] Where x′ is the numerical value of the variable after dimension unification; x is the original numerical value of the variable; x mean and x std These are the mean and standard deviation of the original variables, respectively.

[0090] After unifying the dimensions as described above, the algorithm input data is obtained as data=(f1,f2,f3,l1,l2,l3,n1,n2,n3)′.

[0091] S22. The ultra-large neural network model is trained using the dimensional unified data, and residual connection and batch normalization operations are used during the training phase to obtain an ultra-large neural network for predicting power outages of overhead lines.

[0092] The loss function used for model training is the binary cross-entropy loss function based on data label supervision, expressed by the following formula:

[0093]

[0094] In the formula: y i p represents the true fault label of sample i, with 1 for positive class and 0 for negative class. i Let N represent the probability that sample i is predicted to be of the positive class, and N represent the total number of samples.

[0095] It should be noted that the training dataset constructed in this embodiment includes meteorological feature data, geographical feature data, and power grid feature data. The meteorological features include the maximum intensity of strong convection radar, the magnitude of lightning current, and the number of lightning return strokes, which can be used to predict overhead line power outages under strong current weather conditions.

[0096] In one embodiment, such as Figure 5 As shown, the ultra-large neural network model includes a first input layer, a first backbone network, and a first prediction network. The output features of the first input layer serve as the input of the first backbone network, and the output features of the first backbone network serve as the input of the first prediction network.

[0097] The first backbone network includes a multi-layer neural network, with adjacent layers connected by a residual network. The first prediction network includes a fully connected layer and a softmax classifier, with the output of the fully connected layer connected to the softmax classifier.

[0098] The first backbone network has a depth of 18 layers, with 256 neurons in each layer.

[0099] It should be noted that the power outage prediction model of the ultra-large-scale neural network designed for the power outage prediction task in this embodiment has a first backbone network with a depth of 18 layers, each layer has 256 neurons, and the network has 1.17 million parameters.

[0100] Considering the depth and width of ultra-large neural network models, residual networks are used to connect adjacent layers in the neural network structure. This allows the later layers of a deep network to learn identity mapping models, which degenerate into shallow networks. This greatly reduces the optimization difficulty for deep networks. By using residual connections and batch normalization operations during the training phase, the training convergence is accelerated.

[0101] In one embodiment, reference is made to Figure 6 The lightweight neural network model includes a second input layer, a second backbone network, and a second prediction network. The output features of the second input layer serve as the input of the second backbone network, and the output features of the second backbone network serve as the input of the second prediction network.

[0102] The second backbone network includes a multi-layer neural network, with adjacent layers connected by a residual network. The second prediction network includes a fully connected layer and a softmax classifier, with the output of the fully connected layer connected to the softmax classifier.

[0103] The second backbone network has a depth of 5 layers, with 64 neurons in each layer.

[0104] It should be noted that, considering the actual deployment environment, this embodiment designs a lightweight neural network StudentNet with a depth of D=5 and N=64 features per layer, and introduces residual connections in each layer to reduce training difficulty. The lightweight neural network model has 20,000 parameters, which is 60 times more compressed than the ultra-large neural network model.

[0105] In one embodiment, the loss function of the lightweight neural network model is:

[0106] L = L bce +λL MKD_F

[0107] Among them, L bce L is a binary cross-entropy loss function based on data label supervision. MKD_F The loss function for multi-layer early knowledge fusion distillation is λ, where λ is the weight.

[0108]

[0109]

[0110] Among them, y i p represents the true fault label of sample i, with 1 for positive class and 0 for negative class. i Let represent the probability that sample i is predicted as positive, N represent the total number of samples, i is the layer index of the lightweight neural network model, j is the index of the layers of all ultra-large neural network models before the i-th layer, and D is the knowledge distillation loss function. These are the output features of the intermediate layer of the lightweight neural network model. These are the output features of the intermediate layers of the aforementioned large neural network model.

[0111] Based on the structure of this distillation model, this embodiment proposes a corresponding multi-layer early knowledge fusion distillation loss function, enabling the model to be iteratively optimized using the gradient descent method.

[0112] It should be noted that the knowledge distillation loss function is used to measure the distance between the output of a large neural network model and the output of a lightweight neural network model. In this embodiment, the temperature coefficient of the distillation loss function D is set to 10.

[0113] It should be noted that if deep information from a large neural network model is added to the shallow layers of a lightweight neural network model for supervision, the performance of the resulting lightweight neural network model will decrease. Therefore, this embodiment integrates the output of the first i layers of the large neural network model (i.e., the early knowledge of the large neural network model) to guide the learning of the i-th layer features of the lightweight neural network model. This ensures that the student model is not affected by the deep features of the teacher model in the early stages, achieving better performance. Experiments show that the supervision strategy proposed in this embodiment, namely the multi-layer early knowledge fusion mechanism, can yield a higher-performance lightweight model.

[0114] Furthermore, this embodiment implements the above model training process using PyTorch, training the large-scale model TeacherNet and the lightweight model StudentNet using a training set of 900,000. Ultimately, the StudentNet model achieves 90% precision and 75% recall on a validation set of 100,000.

[0115] To illustrate the advantages of the early-stage knowledge fusion mechanism proposed in this embodiment, a comparative experiment was conducted. StudentNet was trained directly on a 900,000-pixel training set, achieving a precision of 90% and a recall of 40%, a difference of 35%. Therefore, this embodiment demonstrates significant advantages in lightweight model algorithm training.

[0116] It should be noted that, in response to the problem of balancing algorithm performance and model size when deploying deep learning models for predicting overhead power outages during severe convective weather, this embodiment proposes a lightweight method for predicting overhead power outages using a knowledge distillation mechanism based on multi-layer early knowledge fusion. This method ensures both the time required for actual deployment and a significant improvement in model performance under a certain capacity.

[0117] In addition, such as Figure 7 As shown, another embodiment of the present invention proposes a lightweight system for predicting overhead line power outages using a neural network, the system comprising:

[0118] Module 10 is used to acquire overhead line power outage datasets;

[0119] The first training module 20 is used to train a large neural network model on the overhead line power outage dataset;

[0120] The supervision module 30 is used to take the overhead line power outage dataset as input to the lightweight neural network model and fuse the output features of the first i layers of the ultra-large neural network model to supervise the output features of the i-th layer of the lightweight neural network model.

[0121] Knowledge distillation module 40 is used to fit the output features of the last layer of the lightweight neural network model to the output features of the last layer of the ultra-large neural network model, and to perform knowledge distillation on the lightweight neural network model to obtain the target neural network model.

[0122] This embodiment utilizes knowledge distillation to transfer the predictive power of a large model to a lightweight model, achieving a better balance between algorithm performance and model size. This allows the overhead power outage prediction neural network model to be deployed on a central computing cluster for power outage prediction. Furthermore, to enable the lightweight neural network model to learn more useful knowledge, an early knowledge fusion mechanism is designed. This mechanism uses the outputs of the large neural network model in multiple early stages to supervise the outputs of the student model at each stage. By reducing the learning difficulty of the lightweight model through early knowledge fusion distillation, the lightweight model can be trained better, resulting in a higher-performance lightweight model.

[0123] In one embodiment, the overhead line outage data set includes meteorological feature data, geographical feature data, and power grid feature data;

[0124] The meteorological characteristic data includes the maximum intensity of the severe convective radar, the lightning current value, and the number of lightning return strokes.

[0125] The geographic feature data includes elevation, slope, and aspect;

[0126] The power grid characteristic data includes the number of overloads, the number of heavy loads, and the number of defects.

[0127] It should be noted that the training dataset constructed in this embodiment includes meteorological feature data, geographical feature data, and power grid feature data. The meteorological features include the maximum intensity of strong convection radar, the magnitude of lightning current, and the number of lightning return strokes, which can be used to predict overhead line power outages under strong current weather conditions.

[0128] In one embodiment, the ultra-large neural network model includes a first input layer, a first backbone network, and a first prediction network, wherein the output features of the first input layer serve as the input of the first backbone network, and the output features of the first backbone network serve as the input of the first prediction network.

[0129] The first backbone network includes a multi-layer neural network, with adjacent layers connected by a residual network. The first prediction network includes a fully connected layer and a softmax classifier, with the output of the fully connected layer connected to the softmax classifier.

[0130] The first backbone network has a depth of 18 layers, with 256 neurons in each layer.

[0131] In one embodiment, the lightweight neural network model includes a second input layer, a second backbone network, and a second prediction network, wherein the output features of the second input layer serve as the input of the second backbone network, and the output features of the second backbone network serve as the input of the second prediction network.

[0132] The second backbone network includes a multi-layer neural network, with adjacent layers connected by a residual network. The second prediction network includes a fully connected layer and a softmax classifier, with the output of the fully connected layer connected to the softmax classifier.

[0133] The second backbone network has a depth of 5 layers, with 64 neurons in each layer.

[0134] It should be noted that, considering the actual deployment environment, this embodiment designs a lightweight neural network StudentNet with a depth of D=5 and N=64 features per layer, and introduces residual connections in each layer to reduce training difficulty. The lightweight neural network model has 20,000 parameters, which is 60 times more compressed than the ultra-large neural network model.

[0135] In one embodiment, the loss function of the ultra-large neural network model adopts the binary cross-entropy loss function based on data label supervision, and the formula is expressed as:

[0136]

[0137] In the formula: y i p represents the true fault label of sample i, with 1 for positive class and 0 for negative class. i Let N represent the probability that sample i is predicted to be of the positive class, and N represent the total number of samples.

[0138] In one embodiment, the loss function of the lightweight neural network model is:

[0139] L = L bce +λL MKD_F

[0140] Among them, L bce L is a binary cross-entropy loss function based on data label supervision. MKD_F The loss function for multi-layer early knowledge fusion distillation is λ, where λ is the weight.

[0141]

[0142]

[0143] Among them, y i p represents the true fault label of sample i, with 1 for positive class and 0 for negative class. iLet represent the probability that sample i is predicted as positive, N represent the total number of samples, i is the layer index of the lightweight neural network model, j is the index of the layers of all ultra-large neural network models before the i-th layer, and D is the knowledge distillation loss function. These are the output features of the intermediate layer of the lightweight neural network model. These are the output features of the intermediate layers of the aforementioned large neural network model.

[0144] It should be noted that if deep information from a large neural network model is added to the shallow layers of a lightweight neural network model for supervision, the performance of the resulting lightweight neural network model will decrease. Therefore, this embodiment integrates the output of the first i layers of the large neural network model (i.e., the early knowledge of the large neural network model) to guide the learning of the i-th layer features of the lightweight neural network model. This ensures that the student model is not affected by the deep features of the teacher model in the early stages, achieving better performance. Experiments show that the supervision strategy proposed in this embodiment, namely the multi-layer early knowledge fusion mechanism, can yield a higher-performance lightweight model.

[0145] It should be noted that other embodiments or implementation methods of the overhead line power outage prediction neural network lightweight system described in this invention can refer to the above-mentioned method embodiments, and will not be repeated here.

[0146] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0147] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0148] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0149] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0150] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A lightweight method for predicting overhead line power outages using a neural network, characterized in that, The method includes: Obtain the overhead power outage dataset; A large neural network model was trained on the aforementioned overhead line power outage dataset. The overhead line outage dataset is used as input to a lightweight neural network model, and the output features of the first i layers of the ultra-large neural network model are fused together to supervise the output features of the i-th layer of the lightweight neural network model. The output features of the last layer of the lightweight neural network model are fitted to the output features of the last layer of the ultra-large neural network model, and knowledge distillation is performed on the lightweight neural network model to obtain the target neural network model. The loss function of the lightweight neural network model is: L=L bce +λL MKD_F Among them, L bce L is a binary cross-entropy loss function based on data label supervision. MKD_F The loss function for multi-layer early knowledge fusion distillation is λ, where λ is the weight. Among them, y i p represents the true fault label of sample i, with 1 for positive class and 0 for negative class. i Let represent the probability that sample i is predicted as positive, N represent the total number of samples, i is the layer index of the lightweight neural network model, j is the index of the layers of all ultra-large neural network models before the i-th layer, and D is the knowledge distillation loss function. These are the output features of the intermediate layer of the lightweight neural network model. These are the output features of the intermediate layers of the aforementioned large neural network model.

2. The lightweight method for predicting overhead line power outages using a neural network as described in claim 1, characterized in that, The overhead line power outage data set includes meteorological feature data, geographical feature data, and power grid feature data. The meteorological characteristic data includes the maximum intensity of the severe convective radar, the lightning current value, and the number of lightning return strokes. The geographic feature data includes elevation, slope, and aspect; The power grid characteristic data includes the number of overloads, the number of heavy loads, and the number of defects.

3. The lightweight method for predicting overhead line power outages using a neural network as described in claim 1, characterized in that, The process of training a large neural network model on the overhead power outage dataset includes: The overhead line power outage dataset is subjected to dimension unification processing to obtain dimension-unified data; The ultra-large neural network model is trained using the dimensional uniform data, and residual connection and batch normalization operations are used during the training phase to obtain an ultra-large-scale neural network for predicting overhead line power outages. The loss function used for model training is the binary cross-entropy loss function based on data label supervision, expressed by the following formula: In the formula: y i p represents the true fault label of sample i, with 1 for positive class and 0 for negative class. i Let N represent the probability that sample i is predicted to be of the positive class, and N represent the total number of samples.

4. The lightweight method for predicting overhead line power outages using a neural network as described in claim 1, characterized in that, The ultra-large neural network model includes a first input layer, a first backbone network, and a first prediction network. The output features of the first input layer serve as the input of the first backbone network, and the output features of the first backbone network serve as the input of the first prediction network. The first backbone network includes a multi-layer neural network, with adjacent layers connected by a residual network. The first prediction network includes a fully connected layer and a softmax classifier, with the output of the fully connected layer connected to the softmax classifier. The first backbone network has a depth of 18 layers, with 256 neurons in each layer.

5. The lightweight method for predicting overhead line power outages using a neural network as described in claim 1, characterized in that, The lightweight neural network model includes a second input layer, a second backbone network, and a second prediction network. The output features of the second input layer serve as the input of the second backbone network, and the output features of the second backbone network serve as the input of the second prediction network. The second backbone network includes a multi-layer neural network, with adjacent layers connected by a residual network. The second prediction network includes a fully connected layer and a softmax classifier, with the output of the fully connected layer connected to the softmax classifier. The second backbone network has a depth of 5 layers, with 64 neurons in each layer.

6. A lightweight neural network system for predicting power outages on overhead lines, characterized in that, The system includes: The acquisition module is used to acquire overhead line power outage datasets; The first training module is used to train a large neural network model on the overhead line outage dataset. The supervision module is used to take the overhead line power outage dataset as input to the lightweight neural network model and fuse the output features of the first i layers of the ultra-large neural network model to supervise the output features of the i-th layer of the lightweight neural network model. The knowledge distillation module is used to fit the output features of the last layer of the lightweight neural network model to the output features of the last layer of the ultra-large neural network model, and to perform knowledge distillation on the lightweight neural network model to obtain the target neural network model. The loss function of the lightweight neural network model is: L=L bce +λL MKD_F Among them, L bce L is a binary cross-entropy loss function based on data label supervision. MKD_F The loss function for multi-layer early knowledge fusion distillation is λ, where λ is the weight. Among them, y i p represents the true fault label of sample i, with 1 for positive class and 0 for negative class. i Let represent the probability that sample i is predicted as positive, N represent the total number of samples, i is the layer index of the lightweight neural network model, j is the index of the layers of all ultra-large neural network models before the i-th layer, and D is the knowledge distillation loss function. These are the output features of the intermediate layer of the lightweight neural network model. These are the output features of the intermediate layers of the aforementioned large neural network model.

7. The lightweight overhead line outage prediction neural network system as described in claim 6, characterized in that, The overhead line power outage data set includes meteorological feature data, geographical feature data, and power grid feature data. The meteorological characteristic data includes the maximum intensity of the severe convective radar, the lightning current value, and the number of lightning return strokes. The geographic feature data includes elevation, slope, and aspect; The power grid characteristic data includes the number of overloads, the number of heavy loads, and the number of defects.

8. The lightweight overhead line outage prediction neural network system as described in claim 6, characterized in that, The ultra-large neural network model includes a first input layer, a first backbone network, and a first prediction network. The output features of the first input layer serve as the input of the first backbone network, and the output features of the first backbone network serve as the input of the first prediction network. The first backbone network includes a multi-layer neural network, with adjacent layers connected by a residual network. The first prediction network includes a fully connected layer and a softmax classifier, with the output of the fully connected layer connected to the softmax classifier. The first backbone network has a depth of 18 layers, with 256 neurons in each layer.

Images